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1.  INTRODUCTION 


The  Military  faces  unique  challenges  training  medical  personnel  for  deployment  to  combat  zones. 
Pre-deployment,  military  personnel  may  not  be  routinely  performing  the  emergent  procedures  that 
they  will  be  expected  to  perform  in  theater.  The  training  must  be  completed  during  a  short  period 
of  time  and  in  a  “hyper-realistic”  stressful  and  shocking  environment.  An  understanding  of  how 
the  training  modality  impacts  the  translation  of  the  skills  learned  in  training  to  actual  practice,  the 
degree  to  which  those  skills  may  decay  over  time  and  the  impact  of  a  stressful  environment  on 
acquisition  and  retention  of  clinical  skills  is  critically  important.  The  Combat  Casualty  Training 
Initiative  instituted  by  the  Telemedicine  &  Advanced  Technology  Research  Center  (TATRC)  is 
attempting  to  address  several  of  the  complicated  aspects  of  training  military  health  care 
providers.  This  project  developed  a  preliminary  “proof  of  concept”  to  measure  the  impact,  return- 
on-investment,  and  translation  of  the  Department  of  Defense’s  medical  education  training 
objectives.  The  integration  of  simulation  technology  has  augmented  but  not  replaced  the  live 
animal  model  for  many  emergency  procedures  in  the  complex  educational  training  system  of  the 
military  to  prepare  medical  personnel  for  deployment  to  combat  zones.  This  project  describes:"!)  a 
formative  evaluation  to  inform  the  project;  2)  a  proof  of  concept  test  using  system  dynamics  (SD) 
that  can  assist  in  determining  if  simulation  can  replace  the  use  of  live  animals  in  DoD  medical 
readiness  training  if  populated  with  data;  and  3)  an  assessment  of  whether  the  environmental 
stressors  and  distractions  (both  auditory  and  visual)  in  the  simulated  training  environment 
adequately  prepare  the  medical  personnel  for  the  stress  of  the  combat  theater  if  populated. 

The  reverence  for  data  as  the  source  of  knowledge  seems  to  be  the  rule  for  evidence.  In 
this  respect,  system  dynamics  has  much  to  offer.  One  of  its  trademarks  is  “operational  thinking”,  a 
distinct  attitude  that,  unlike  scientific  practices  that  seek  to  understand  the  world  by  means  of 
data  analysis,  seeks  to  transform  systems  in  terms  of  operations.  Operational  thinking  means  to 
recognize  agency  as  the  driving  force  through  which  systems  actually  work:  free  decision  makers 
invent  their  own  future  through  actions  that  shape  and  create  systems;  they  do  not  obey 
laws  to  be  discovered  by  observing  the  past.  As  a  standard  example,  if  we  observe  several 
college  aged  women,  we  cannot  use  their  past  behavior  to  make  a  good  estimate  of  future 
behavior  when  it  comes  to  marriage  and  children.  As  a  rule,  education  staves  off  age  of  marriage 
and  age  of  first  birth.  However,  the  conundrum  is  that  most  of  these  college  aged  women  will  go 
on  to  have  families  with  both  spouses  and  children.  Thus,  the  past  cannot  predict  the  future  but 
looking  at  the  behaviors  of  other  women  who  went  to  college  does  inform  the  system  and  helps  to 
explain  and  estimate  future  behaviors.  System  dynamics  models  provide  intelligible  explanations 
that  meet  such  agency.  Redesigning  and  improving  human  systems  require  then  the  capacity  to 
realize  and  transform  operational  arrangements,  that  is,  to  understand  and  change  decision 
processes  and  purposeful  actions.  System  dynamics  modeling  relies  on  both  quantitative  and 
qualitative  data  and  allows  us  to  run  simulations  on  policies  and  protocols.  It  derives  information 
to  make  estimates  not  from  study  subjects  past  behavior  but  from  experts  with  similar 
experiences.  The  “building  blocks”  of  SD,  “operational  thinking”  rarely  makes  is  included  in  SD 
modeling.  SD  modelers  usually  build  operational  models  but  this  special  characteristic  is  hardly 
spelled  out  to  make  the  most  of  it.  Our  purpose  in  using  SD  modeling  to  determine  whether  the 
DoD  can  safely  reduce  live  tissue  use  and  replace  it  with  simulators,  was  to  uncover  the 
significance  of  operational  thinking  for  modeling  social  systems.  Thinking  in  terms  of  operations 
represents  a  distinct  paradigmatic  posture,  different  from  the  widespread,  scientific  paradigm  that 
seeks  to  understand  human  systems  through  data  analysis.  This  latter  path  conveys  opportunities 
but  also  risks.  This  study  shows  why  operational  thinking  is  indeed  a  distinct  way  of  “thinking”,  a 
way  of  seeing.  Instead  of  developing  knowledge  by  observation  of  live  tissue  training,  we 
formulate  more  general  statements,  the  production  of  knowledge  through  operational  modeling 
does  not  rest  on  data  in  order  to  bring  understanding,  (see  example  of  college  age  women  having 
families  above).  Rather,  it  relies  on  the  generation  of  dynamic  hypotheses  such  as  The  DoD 
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can  safely  reduce  the  use  of  animals  in  medical  training.  We  can  explain  the  performance  of  a 
system  based  on  its  structure  that  is  formed  by  operations.  In  turn,  thinking  in  terms  of  operations 
in  social  systems  means  to  think  in  terms  of  actual  decision  making  processes  continuously 
carried  out  by  free  actors.  From  this  perspective,  the  performance  of  a  social  system  is 
recognized  as  the  result  of  human  action. 

The  recognition  of  operational  thinking  as  a  distinct,  separable  and  intrinsic  element  of  SD 
represents  various  opportunities.  The  prominence  of  understanding  over  data  analysis  is  a  key 
example.  It  is  not  uncommon  to  hear  that  “rigor”  means  adequate  data  analysis  of  “empirical 
evidence”  for  justifying  model  building  and  valid  results.  SD  models  are  certainly  empirical  since 
they  are  built  from  sense  experience  (rather  than  from  pure  theory  or  logic)  and  are  checked 
against  experiential  information;  but  this  is  not  equivalent  to  stating  that  SD  models  are  built 
inductively  from  observed  data.  The  current  resurgence  of  data-driven  research  just  boosts 
pressure. 

Finally,  this  project  generated  4)  a  preliminary  return  on  investment  (ROI).  An  ROI  is  the  benefit 
to  an  investor  resulting  from  an  investment.  In  our  case,  that  investment  is  the  purchase  and  care 
of  maintaining  animals  or  the  costs  of  purchasing  sophisticated  simulators  and  replacing  the 
tissue  of  the  simulator  after  use.  An  ROI  that  is  large  means  the  investment  gains  compare 
favorably  to  investment  cost.  As  a  performance  measure,  ROI  is  used  to  evaluate  the  efficiency  of 
an  investment  or  to  compare  the  efficiency  of  a  number  of  different  investments  (animals,  animal 
food,  animal  housing,  caretakers,  simulators,  replacement  parts,  etc).  In  purely  economic  terms, 
it  is  one  way  of  considering  profits  in  relation  to  invested  investments.  The  ROI  can  also  be 
plugged  into  the  SD  model  to  better  inform  the  impact  of  the  various  modes  of  training  along  with 
research  results  produced  by  other  investigators  from  the  Consortium  of  Simulation  vs.  Live 
Animal  Training. 


2.  KEYWORDS 


Simulation  training 
Procedural  training 
Live  Patient  Model 
Live  tissue  training 
Stress  Inoculation 
Combat  medic  readiness 
Combat  casualty  care 
Military  preparedness 
System  Dynamics  Modeling 
Return  on  Investment 


3.  ACCOMPLISHMENTS 

Major  Goals  of  Project: 

a.  Formative  Evaluation:  The  goal  of  the  formative  evaluation  was  to  provide  “useful 

feedback"  to  the  project  team  conducting  other  activities:  The  DoD,  trainers,  trainees,  and 
the  research  consortium  at  large.  Depending  on  the  training  site  and  the  type  of  training 
being  addressed,  the  process  involved  these  stages:  the  formulation  of  the  issues  with 
live  animal  and  simulation  training;  the  broad  conceptualization  of  the  major  alternatives 
that  might  be  considered;  the  detailing  of  these  alternatives  and  their  potential 
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implications;  a  review  of  the  alternatives.  As  this  project  is  only  partially  empirical,  we  did 
not  make  recommendations  on  alternative  or  better  training  models  but  rather  developed 
a  proof  of  concept  that  attempted  to  answer  these  questions.  The  formative  research 
addressed  several  major  questions: 

1 )  What  is  the  definition  and  scope  of  the  problem  or  issue?  Formulating  and 
conceptualizing  methods  included  brainstorming,  focus  groups,  interviews,  nominal  group 
techniques,  stakeholder  analysis,  and  concept  mapping.  While  it  is  understood  that  the 
DoD  would  like  to  reduce  the  use  of  live  animals  when  possible  in  medical  training,  a  full 
understanding  and  a  more  formal  definition  of  this  problem  is  needed. 

2)  Where  is  the  problem  and  how  big  or  serious  is  it?  The  most  common  method  used 
here  is  a  "needs  assessment,"  which  includes  a  review  of  existing  data  sources, 
interviews  of  constituent  populations  including  researchers,  program  directors,  trainers 
and  trainees,  and  focus  groups. 

3)  How  should  the  training  program  be  delivered  to  address  the  problem?  Some  of  the 
methods  already  listed  apply  here  and  we  will  also  use  flow  charts  to  answer  this 
question.  Drs.  Evans  and  Post  directly  observed  the  training  at  a  variety  of  sites  and  had 
unstructured  conversations  with  a  variety  of  stakeholders.  We  observed  training  within 
their  context  for  a  broader  understanding  of  the  issues.  We  observed  sampled  trainings 
and  people  rather  than  becoming  immersed  in  the  entire  context.  The  unstructured 
conversations  occurred  with  the  PI  consortium,  a  sampling  of  trainers  and  trainees  and 
finally  with  subject  matter  experts  in  simulation  and  the  medical  procedures  being  taught. 
In  summary,  much  of  this  information  is  understood  by  the  DoD  and  the  Consortium, 
however,  in  order  for  the  project  team  to  develop  a  robust  proof  of  concept  for  the  system 
dynamics  modeling  and  the  return  on  investment,  is  was  necessary  for  us  to  conduct  a 
small  independent  formative  evaluation  to  inform  the  rest  of  this  project. 

Proof  of  Concept  System  Dynamics  Model:  System  dynamics  is  a  way  to  understand 
systems  and  how  they  change  with  a  specific  focus  on  identifying  and  evaluating  the 
impact  of  feedback  loops,  accumulations  (stocks  and  flows),  and  the  short  term  and  long 
term  consequences  of  different  decisions  (Astrom  &  Murray,  201 0;  Forrester  &  Senge, 
1978;  Richardson,  2011).  In  system  dynamics,  this  is  done  by  building  and  analyzing 
informal  causal  maps  and  formal  computer  models  of  a  system  that  can  be  simulated. 
Unique  to  system  dynamics  is  the  potential  to  involve  diverse  stakeholders,  key 
informants,  and  other  experts  in  the  process  of  building  and  analyzing  the  results  of  the 
model  using  participatory  methods  such  as  group  model  building  (GMB).  GMB  walks 
participants  through  all  stages  of  of  building  a  system  dynamics  model  using  a  series  of 
structured  small  group  exercises,  from  conceptualizing  the  problem  and  elements  of  the 
system  through  formulation,  analysis,  and  implementation  of  the  results.  For  this  project, 
the  primary  aim  was  to  develop  a  "proof  of  concept"  system  dynamics  model  of  the 
implementation  of  stress  simulations  for  increasing  resiliency  using  group  model  building. 
The  goal  was  to  illustrate  both  how  this  approach  can  be  used  and  evaluate  its  feasibility 
and  utility  for  informing  military  policies  on  the  use  of  stress  simulations  (e.g.,  What 
should  be  the  frequency  of  trainings  based  on  trauma  simulations  given  the  decay  of 
resiliency  over  time?  How  would  differences  in  the  backgrounds  of  soldiers  affect  the 
overall  resiliency  of  the  military  personnel?).  The  specific  steps  for  this  phase  of  the  study 
were:  Recruit  and  convene  a  core  modeling  team  including  members  of  the  research 
team  and  "gate  keepers"  for  the  military  to  design  the  group  model  building  (GMB) 
workshops  Vennix  &  Vennix,  1996).  The  main  deliverables  are  a  system  dynamics’  seed 
models  for  introducing  system  dynamics,  analysis  of  the  model  and  assessment  of  the 
feasibility  and  utility  of  this  approach  as  shown  below  as  stock  and  flow  diagrams. 
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b.  Assessment  of  Environmental  Stressors-  Environmental  stressors  are  known  to 
impact  performance.  Dr.  Stephen  Barnes’  Grant  W81XWH-1 1-2-01 55#1 1322004 
sponsored  by  the  Department  of  Defense  Telemedicine  and  Advanced  Research  Center 
(TATRC)  used  sweat  measurement  as  a  potential  indicator  of  a  stressful  response  to  the 
combat  training  environment.  This  was  accomplished  by  measuring  electrodermal 
activity  during  didactic  and  hands-on  training.  Training  participants  completed  a  survey 
regarding  perceived  value  of  the  training.  A  goal  of  the  current  study  was  to  incorporate 
these  survey  results  and  interviews  conducted  at  the  Combat  Casualty  Training 
Consortium  meeting  at  University  of  Central  Florida’s  Institute  for  Simulation  and  Training 
into  the  system  dynamics  concept  model  for  further  direct  comparison  of  environmental 
stressors  on  live  tissue  training  and  simulation-based  training  for  combat  casualty  care. 
Other  potential  measures  of  “being  environmentally  stressed”  was  the  use  of  cortisol 
levels,  blood  pressure,  temperature,  etc. 


c.  Preliminary  Return  on  Investment  Overview:  Dr.  Ted  Miller,  a  health  economist 

developed  a  thorough  analysis  plan  for  a  return  on  investment  (ROI)  analysis  associated 
with  simulation  training  versus  live-animal  training.  The  analysis  plan  described  four 
major  steps  to  be  undertaken  in  a  ROI  study  in  the  context  of  System  Dynamics:  (1) 
describe  each  training  alternative;  (2)  determine  staff  hours,  supplies,  travel,  and  other 
costs  associated  with  each  training  approach;  (3)  calculate  costs  and  ROI,  and  (4) 
communicate  and  document  findings.  Dr.  Miller  prepared  the  first  step  by  undertaking  a 
study  of  System  Dynamics  modeling.  He  then  read  and  interviewed  clinicians  to  fully 
understand  the  two  training  approaches  and  how  their  costs  can  be  modeled  within  a 
System  Dynamics  framework. 

The  second  step  documented  the  actual  costs  associated  with  each  approach.  Both 
types  have  associated  staff  time  for  trainers  and  for  trainees.  Live-tissue  training  has  an 
additional  cost  for  procuring,  housing,  and  disposing  of  animals  used  in  training. 
Simulation  training  carries  a  cost  for  the  simulation  units.  If  the  simulation  training  is  given 
by  contractors  offsite,  then  their  costs  must  be  obtained  or  estimated.  We  expect  that 
many  gaps  will  remain,  and  so  Dr.  Miller  also  studied  the  clinical  and  cost  datasets  that 
could  provide  necessary  information.  Inputs  of  machinery,  travel,  and  staff  hours  came 
from  discussions  with  experienced  trainers,  for  example,  while  costs  were  extracted  from 
DoD  databases  or  from  procurement  records.  The  cost  of  staff  and  student  time  was 
based  on  the  typical  distribution  of  job  types  they  represent  (e.g.,  physician,  physician 
assistant  /  nurse  practitioner,  registered  nurse,  medic)  and  average  labor  costs  for  those 
job  types.  Dr.  Miller  develop  used  interviews  with  the  Pis  to  answer  questions  that  cannot 
be  addressed  with  administrative  data.  The  third  ROI  step  was  to  pull  together  results 
from  the  first  two  steps  to  calculate  the  costs  of  each  training  approach  and  the  ROI. 
During  the  initial  proof-of-concept  study,  Dr.  Smith  and  his  team  will  develop  a  clear, 
detailed  Analysis  Plan  for  calculating  each  of  these  components.  Dr.  Miller  also  made 
methodological  decisions  such  as  selecting  a  method  for  adjusting  costs  for  inflation, 
determining  the  discount  rate  for  future  costs  and  benefits,  and  choosing  the  perspective 
of  the  analysis.  Dr.  Miller  will  use  the  initial  proof-of-concept  study  to  develop  a 
presentation  plan.  He  will  confer  with  DoD  officials  and  Consortium  researchers  to 
determine  the  tables  and  graphics  that  will  provide  the  most  value  when  presenting  ROI 
results.  See  COI  below. 


Accomplishments  Under  Goals: 
a.  Formative  Evaluation 

Drs.  Evans  and  Post  attended  the  Missouri  consortium  of  researchers  addressing  our 
hypothesis  about  whether  or  not  it  is  possible  to  safely  reduce  the  use  of  live  tissue  while 
increasing  the  use  of  simulators  and  to  have  similar  outcomes  in  terms  of  skills,  skill 
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decay  and  skill  retention.  They  were  present  when  all  the  research  results  were 
discussed  and  there  was  also  a  phase  of  the  visit  whereby  Dr.  Evans  described  her  role 
in  simulations  and  to  determine  if  the  team  of  experts  felt  live  tissue  could  be  reduced 
safely.  This  resulted  in  a  detailed  discussion  of  the  pros  and  cons  of  using  each.  Copious 
notes  were  collected  by  Pis  Evans  and  Post  describing  the  studies  and  research 
questions  under  review. 

The  Combat  Casualty  Training  Consortium  -  Dr  Stephen  Barnes,  Principal  Investigator, 
University  of  Missouri  February  18-19,  2014.  Pis  Evans  and  Post  presented  an 
Introduction  to  System  Dynamics  Modeling  with  an  SD  model  example  of  decision 
making  in  faculty  tenure  related  to  female  gender,  the  use  of  checklists  and  ‘training  to 
competency’  in  procedural  simulation  and  a  discussion  on  health  economics.  The 
objective  was  to  develop  a  conceptual  model  that  can  be  used  by  the  Consortium  to 
support  conclusions  and  recommendations  to  the  DoD.  Boundaries  included:  ‘live  tissue 
vs  simulation,’  ‘stress  inoculation,’  ‘self  efficacy,’  ‘skills  transfer  to  the  clinical  setting  of 
combat,’  and  ‘skills  decay  over  time.’ 

On  the  topic  of  live  tissue  vs.  simulation,  the  discussion  focused  on  exploring  the 
following  questions: 

•  What  are  the  advantages  of  simulation  training? 

•  What  are  advantages  of  a  live  tissue  model? 

•  Are  advantages  and  disadvantages  specific  to  individual  procedures? 

•  For  animal  models: 

o  How  were  species  selected  for  each  procedure?  Were  there  issues 
related  to  anatomic  accuracy  of  the  animal  model  compared  to 
humans? 

•  For  task  trainers: 

o  Did  the  trainers  fail?  Was  the  fidelity  of  the  partial  task  trainers 
adequate? 

•  Since  testing  occurred  on  a  simulator,  is  it  possible  the  results  were  influenced 
by  the  simulation  trained  group  being  more  comfortable  with  a  simulator?  How 
was  this  addressed? 

With  regard  to  stress  inoculation,  the  discussion  focused  on: 

•  How  was  electro-dermal  activity  chosen  as  the  measurement  of  stress? 

•  Were  other  physiologic  measurements  considered? 

•  Did  you  evaluate  for  a  maximum  level  of  stress  that  might  be  detrimental  to 
retention? 

When  working  with  groups  unfamiliar  with  SD  modeling,  modelers  need  a  quick  way  to 
introduce  the  iconography  of  the  approach  and  some  of  its  framing  assumptions,  which 
are  often  called  concept  models.  The  concept  model  represents  our  approach  to  early 
view  of  the  problem  in  its  systemic  context.  (See  Appendix  1) 

In  this  concept  model,  “Combat  Medics”  is  the  fundamental  stock  and  the  “Total  Medic 
Skill”  is  the  underlying  attribute.  Note  that  “Total  Medic  Skill”  is  a  net  function  of  “Skill 
Increase  by  Experience”,  “Skill  Increase  by  Appointment”,  “Skill  Increase  by  Training”, 
and  “Skill  Loss  in  Turnover”  (i.e.,  medic  attrition). 


b.  Proof  of  Concept  System  Dynamics  Model:  Achieving  desired  levels  of  combat 

casualty  care  skills  and  resiliency  under  combat  stress  using  different  training  modalities 
(e.g.,  live  tissue  training,  simulated  medical  procedures)  is  a  dynamic  resource  allocation 
problem.  Essential  features  of  this  problem  are  1)  training  modalities  vary  in  their 
effectiveness  and  rate  of  skill  decay  by  the  type  of  skill,  2)  experienced  providers  are  a 
critical  resource  for  delivering  training,  and  3)  training  is  required  to  maintain  existing 
skills  for  trained  and  experienced  providers.  This  contributes  to  a  set  of  variables 
affecting  each  other  through  one  or  more  feedback  mechanisms  that  pose  a  set  of 
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complex  tradeoffs  about  how  to  best  allocate  scare  resources  to  maximize  the  quality  of 
care. 

To  address  this  dynamic  resource  allocation  problem,  we  developed  a  prototype  system 
dynamics  computer  simulation  model.  System  dynamics  is  a  method  for  understanding 
the  behavior  of  complex  systems  using  informal  maps  and  formal  models  with  computer 
simulations  from  a  feedback  perspective  (1).  As  such,  system  dynamics  (SD)  provides  a 
novel  way  to  gain  insight  into  systems  and  formulate  better  policies.  The  purpose  of  the 
combat  casualty  care  system  dynamics  prototype  was  to  provide  a  “proof  of  concept”  for 
how  system  dynamics  computer  modeling  and  simulation  might  be  used  to  inform 
decisions  about  how  to  develop  and  allocate  a  variety  of  training  modals  to  maximize 
quality  of  emergency  medical  care  in  combat  settings. 
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To  inform  the  modeling  process,  we  developed  a  preliminary  model,  conducted  group 
model  building  workshops,  and  drew  on  key  informant  interviews  with  experts  familiar 
with  combat  medicine  training.  Group  model  building  (GMB)  is  a  participatory  methods  for 
involving  experts  and  stakeholders  in  the  process  of  conceptualizing  and  building  a 
system  dynamics  models  (e.g.,  2,  3).  We  conducted  two  90-minute  GMB  workshops,  one 
with  emergency  medicine  providers  with  combat  experience  and  a  second  session  of 
providers  without  combat  experience.  The  results  of  the  resulting  diagrams  were 
synthesized,  and  then  reviewed  as  part  of  a  key  informant  process  led  by  the  larger 
research  team. 

The  initial  prototype  was  developed  using  Vensim  PLE,  but  eventually  rebuilt  and  revised 
using  Stella  Architect  1.1,  which  supports  the  development  on  online  web  interfaces  for 
experimenting  with  the  simulation  model. 

Figure  1  provides  a  high  level  overview  of  the  final  prototype  Combat  Casualty  Care 
Model  version  4.0.1  depicting  some  of  the  major  feedback  loops  and  the  basic  “aging 
chain”  for  providers  (the  variables  in  boxes  including  untrained  providers,  trained 
providers,  and  experienced  providers).  The  double  lines  with  the  circular  valves  represent 
rates  of  change  (e.g.,  recruiting,  training,  gaining  experience)  and  the  clouds  represent 
the  sources  and  sinks  that  define  the  material  and  information  boundaries  of  the  system. 
The  single  lines  with  arrow  heads  represent  causal  links  where  the  polarity  of  the  sign 
represents  the  direction  of  association.  So,  for  example,  the  greaterthe  gap  in  skills  for 
trained  providers,  the  more  trainers  are  needed  indicating  a  positive  link.  Similarly,  the 
higher  the  proportion  of  experienced  providers  allocated  to  training,  the  slower  the  net 
change  in  allocation  since  there  are  fewer  experienced  providers  left  to  allocate  to 
training  indicating  a  negative  relationship. 


Figure  1  Conceptual  overview  of  Combat  Casualty  Care  system  dynamics  model 
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The  actual  model  disaggregates  skills  and  resiliency  under  stress  by  the  training  modality 
(training  1  and  training  2),  practice  experience,  and  experience  level  of  the  providers 
(untrained,  trained,  and  experienced).  Skill  and  resiliency  developed  through  training  are 
considered  vulnerable  to  skills  decay,  while  skill  and  resiliency  developed  through 
practice  experience  are  considered  more  robust  and  permanent.  The  final  model  has  14 
stocks  or  state  variables,  133  initialization  equations,  and  87  runtime  equations. 

The  user  sets  the  characteristics  of  training  1  and  training  2  including  the  effectiveness 
each  training  for  developing  skills  and  resiliency  under  stress  (a  1 00-point  scale  from 
0=no  skills  and  resiliency  to  100=maximum  skills  and  resiliency)  for  each  level  of  training 
(untrained,  trained,  and  experienced),  rate  of  skill  decay,  and  cost  in  addition  to  the 
proportion  receiving  training  1,  and  hence  also  the  proportion  receiving  training  2  (1- 
proportion  receiving  training  1).  This  allows  the  user  to  consider  a  wide  variety  of 
situations.  Importantly,  the  comparisons  are  only  with  respect  to  one  kind  of  medical 
procedure.  This  model  does  not  represent  all  the  complex  interactions  that  can  occur 
when  performing  one  medical  procedure  is  contingent  on  competencies  related  to 
another  medical  procedure  (this  is  something  that  could,  however,  be  captured  in  a  much 
more  sophisticated  research  model). 

We  have  developed  a  simple,  demonstration  web-interface  for  the  model  to  illustrate  the 
possibilities.  This  provides  a  brief  introduction  to  the  model  and  interface,  takes  the  user 
through  a  parameter  setting  page  for  the  two  training  modalities,  and  then  lets  the  user 
conduct  a  series  of  experiments  to  test  various  implications  of  allocating  training,  setting 
goals  for  maintaining  skills  and  resiliency  for  trained  and  experienced  providers,  and 
addressing  turnover.  Note  that  the  model  starts  initially  in  a  dynamic  equilibrium  that  is 
calculated  internally  based  on  user  supplied  values.  To  see  a  dynamic  pattern  as  shown 
in  Figure  2,  one  must  first  set  a  goal  to  increase  (or  decrease)  the  desired  level  of  change 
for  trained  providers  and/or  experienced  providers. 
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Proportion  of  training  for  untrained 
providers  using  training  1 

0.0  0.50  1.0 

- -  0 

Proportion  of  training  for  trained 
providers  using  training  1 

0.0  0.50  1.0 

•"O - ■ - - 

Proportion  of  training  for 
experienced  providers  using  training 
1 

0.0  0.50  1.0 

Q 


Prop  change  in  goal  for  trained 
providers 

0.50  1.3  2.0 

*  - - a 

Prop  change  in  goal  for  experienced 
providers 


0.50  1.3  2.0 


T urnover  fraction  for  trained 
providers 

0.050  0.40  0.75 

— - - - O 

Turnover  fraction  for  experienced 
providers 

0.050  0.40  0.75 


Make  policy  decisions  about  what  proportion  of  each  type  of  provider  receives  training  1, 
while  the  remaining  will  receive  training  2.  You  can  also  set  the  goals  for  trained  and 
experienced  providers'  skills  and  resiliency  under  stress.  For  example,  setting  the  "prop 
change  in  goal  for  trained  providers"  from  1.0  to  1.5  will  mean  setting  a  goal  to  increase 
skills  and  resiliency  under  stress  by  50%  for  trained  providers.  Lastly,  you  can  also  set  the 
turnover  fraction  for  trained  and  experienced  providers. 

Start  the  simulation  by  starting  "Go  Live"  and  see  the  changes  as  you  make  changes.  To 
top  "Go  Live’,  hit  the  ’Stop  Live"  button.  After  starting  "Go  Live",  you  can  explore  the 
results  in  the  graph  below. 


Average  Skills  and  Resiliency  P9  1  i  ► 


Years 


Avg  skills  of  providers  —2—  Avg  skills  of  trained  providers 

— 3—  Avg  skills  of  experienced  providers 


Go  Live 

Previous 

Live 

Paqo 

Figure  2  Example  of  a  page  from  the  web-interface 


Summary 

The  Combat  Casualty  Care  model  demonstrates  the  basic  feasibility  of  creating  a  system 
dynamics  simulation  model  to  address  the  question  about  how  to  best  allocate  training 
resources  to  maintain  quality  of  emergency  medicine  care.  The  process  of  developing  the 
model  also  generated  a  number  of  insights  into  the  system  and  some  of  the  underlying 
complexities  discussed  during  GMB  workshops  and  key  informant  interviews,  particularly 
with  the  medical  providers  with  combat  experience. 


For  example,  a  recurring  theme  during  the  discussions  when  forced  to  describe  the 
dynamics  around  specific  skills  and  resiliency  with  respect  to  training  modalities.  Live 
tissue  training,  for  example,  varied  in  its  effectiveness  by  the  type  of  procedure,  and  there 
could  be  interactions  between  competencies  for  one  procedure  another  procedure. 

Future  efforts  should  consider  arraying  the  skills  and  resilience  under  stress  by  a  detailed 
list  of  procedures  and  the  potential  interactions  in  a  matrix.  Although  existing  data  may 
not  be  sufficient  to  fully  specify  all  the  parameters,  working  with  expert  key  informants 
would  likely  be  able  to  establish  reasonable  estimates. 


A  second  insight  and  point  for  potential  research  might  focus  on  understanding  more 
precisely  the  relationship  between  experience  and  dynamics  of  skill  decay.  For  example, 
to  what  extent  does  acquiring  a  skill  using  one  training  technique  help  retain  skills  learned 
through  other  training  technique.  In  the  GMB  workshops,  a  frequent  comment  was  the 
effect  that  exposure  to  blood  under  stress  could  help  inoculate  providers  to  stress 
(something  that  was  generalizable  to  any  experience  involving  exposure  to  blood  and 
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perhaps  others).  Similarly,  questions  arose  during  the  modeling  process  about  the  rate  of 
skill  decay  by  experience  level.  Do  trained  providers  with  little  experience  have  greater 
skill  decay  for  a  specific  procedure  than  experienced  provider?  The  answers  to  these 
kinds  of  questions  turned  out  to  have  important  implications  for  the  dynamics  of  average 
skill  and  resiliency  and  would  require  further  study  for  a  more  fully  developed  system 
dynamics  model. 

Lastly,  many  aspects  of  this  structure  may  have  much  more  generalizability  to  other 
areas  where  training  is  often  done  by  experienced  providers,  skills  decay,  performance 
depends  on  successfully  completing  one  more  procedures,  and  performance  is  affected 
by  stress  (e.g.,  other  roles  in  combat). 

c.  Assessment  of  Environmental  Stressors 

For  purposes  of  this  study,  study  subject  “sweating”  was  used  to  measure  the 
environmental  stressors. 


d.  Preliminary  Return  on  Investment:  Traditionally,  training  for  invasive  procedures 

included  either  the  apprenticeship  model  in  which  the  trainee  learns  technical  skills  under 
the  guidance  of  an  expert  or  the  use  of  live  tissue  patients  (LTPs)  to  allow  the  trainee  to 
gain  these  critical  skills.  Dr  Roy  Ziegelstein  of  Johns  Hopkins  University  believes  live 
tissue  patients  give  students  “the  experience  of  being  directly  responsible  for  decisions  in 
life-threatening  situations”  (Fears  201 6).  However,  over  the  past  twenty  years,  the 
development  of  high  fidelity  mannequin  simulators  and  procedural  task  trainers  offers  an 
alternative  method  of  training  medics  for  the  combat  field.  It  has  been  postulated  that  live 
tissue  maximizes  the  realism,  stress  and  emotional  connect  present  in  actual  medical 
emergencies  in  the  combat  setting,  Despite  that  benefit,  improved  simulation  technology 
and  concerns  about  animal  rights  have  led  all  US  medical  schools  to  minimize  surgical 
training  on  living  tissue  (Fears  2016).  The  military,  however,  has  continued  to  use  live 
tissue  models  to  prepare  personnel  to  render  medical  assistance  on  the  battleground.  We 
compared  the  costs  of  combat  casualty  training  using  living  tissue  versus  simulators. 
Simulators  included  high  fidelity  mannequin  simulators,  procedural  task  trainers  and 
hybrid  simulators  (human  volunteer  strapped  to  a  half-body  mannequin). 

METHODS  We  conducted  the  analysis  from  the  US  military’s  perspective.  We  organized 
the  methods  around  4  questions: 

1 .  What  alternatives  will  be  assessed? 

2.  What  are  the  costs  of  each  alternative? 

3.  How  do  the  benefits  differ  between  the  alternatives? 

4.  How  sensitive  are  the  findings  to  assumptions  and  uncertainty? 

Alternatives  Assessed 

1 .  Living  Tissue:  one  session  of  combat  casualty  training  on  goat  tissue. 

2.  Mannequin  Only:  one  session  of  simulator  training  without  a  volunteer  patient 

3.  Hybrid:  one  session  of  simulator  training  with  a  human  providing  half  of  the 
patient’s  body)  with  wounds  on  lower  portion  of  body  (simulator). 

4.  Ride  Along  Supplement:  one  day  of  trainee  ride-along  as  a  paramedic’s  shadow, 
with  the  Emergency  Medical  Service  (EMS)  agency  paid  a  $100  ride-along  fee. 

Provided  each  alternative  training  approach  teaches  the  same  medical  procedures, 
session  length  will  not  vary  between  alternatives  1-3. 

Costs.  Costs  for  live  tissue  were  obtained  from  interviews  with  military  program 
managers.  Simulator  costs  were  provided  from  GSA  schedule.  Hybrid  surgical 
simulations  used  non-commissioned  officers  or  wounded  warriors  who  volunteered  to 
participate.  We  estimated  a  typical  simulation  requires  3  hours  of  the  volunteer’s  time.  In 
June,  2016,  median  annual  non-commissioned  Army  officer  salaries  were  $26,510  for  E4 
and  $28,51 6  for  E5  (wwwl  .salary.com)  for  a  2,000-hour  work-year.  We  used  the  $27,51 3 
average  of  these  medians.  We  added  noncash  compensation  including  health  care, 
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retirement  pay,  child  care  and  free  or  subsidized  food,  housing  and  education.  Those 
supplements  represent  roughly  60  percent  of  total  compensation 
(http://www.goarmy.com/benefits/total-compensation.html),  hourly  compensation 
averages  $34  ((27,51 3/2000)/(1 -0.6)). 

We  stated  all  monetary  estimates  in  2016  dollars.  We  used  the  2.9%  discount  rate 
mandated  by  the  US  Office  of  Management  and  Budget  (2016)  and  US  Department  of 
Defense  (2015)  to  amortize  future  costs. 

Benefits.  All  options  will  yield  the  necessary  medical  skills.  Live  tissue  patient  training 
conceivably  could  prevent  a  new  medical  provider  from  freezing  the  first  time  (s)he 
provides  treatment  during  active  combat.  A  ride  along  supplement  seems  likely  to  have  a 
similar  effect,  and  hybrid  training  may  as  well.  The  effect  with  hybrid  training  may  be 
somewhat  attenuated. 

Sensitivity  Analysis.  We  examined  how  the  costs  varied  when  we  changed  alternate 
assumptions  and  priced  three  competing  simulators. 

RESULTS 

As  Table  1  shows,  a  typical  live  tissue  training  session  requires  an  estimated  $675  in 
disposables  per  trainee,  with  most  of  the  cost  for  live  model  acquisition  and  disposal. 
Because  all  hands-on  training  alternatives  last  equally  long  and  require  one  trainer  per 
trainee,  neither  trainee  nor  trainer  time/costs  will  differ  among  the  alternatives,  so  we  did 
not  cost  them. 

Table  1.  Cost  per  Live  Patient  Model  Session 


Cost  Category 

Cost  per  Trainee 

Live  Model  Purchase  (Goat) 

$275 

T ransfer/Storage  Cost* 

50 

Anesthetization  Cost 

125 

Disposal  Cost 

225 

TOTAL 

$675 

*  Assumes  at  least  4  medical  personnel  are  trained  on  the  same  day. 

Table  2  shows  that  manikin-only  simulations  cost  an  estimated  $452  per  trainee,  only 
slightly  less  than  $472  cost  of  a  hybrid  simulation  assisted  by  a  human.  Both  cost 
substantially  less  per  trainee  than  live  tissue  training.  In  sensitivity  analysis,  if  the  full 
manikin  was  used  in  hybrid  training,  hybrid  training  costs  would  rise  to  $554.  If  the 
manikin  plus  warranty  had  a  7-year  useful  life  rather  than  10  years,  costs  per  trainee 
would  rise  to  $507  for  the  full  manikin  and  $497  for  hybrid  training.  Undiscounted  the 
costs  are  $433  for  the  full  manikin  and  $463  for  the  hybrid.  Using  a  3%  discount  rate 
would  raise  each  manikin  option’s  cost  by  $1 . 

DISCUSSION 

This  analysis  has  limitations.  It  counted  neither  any  gain  in  emotional  readiness  from  live 
tissue  training  nor  harm  to  the  military’s  reputation  and  possible  disruption  of  training 
efforts  associated  with  use  of  live  tissue.  Manikin  costs  were  drawn  from  GSA  schedule. 
They  are  based  on  the  three  mannequins  usually  purchased  for  the  training  analyzed  but 
may  not  reflect  the  exact  options  selected  in  mannequin  purchase.  The  estimated  useful 
mannequin  life  also  is  an  approximation. 

Despite  these  limitations,  it  is  clear  that  simulated  training  costs  less  than  live  tissue 
training.  Hybrid  training  increases  realism  and  emotional  preparation  at  a  relatively 
modest  cost,  especially  if  it  permits  use  of  a  half  manikin  rather  than  a  full-body 
mannequin. 
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Table  2.  Cost  per  Mannequin  Session 


Item 

Price  Range 

Per  Trainee 

MANNEQUIN  ONLY 

Full-body  mannequin  (one  trainee/week,  10  year  life) 

$65,000-$80,000 

$164 

Warranty  (per  trainee  cost  adjusted  for  year  1  free) 

$7, 350/year 

132 

Blood 

$6.40/gallon 

6 

Skin  for  needle  chest  decompression,  and  surgical  cricothyrotomy 

$90/set 

Lubricant 

$1 0/tube 

10 

Partially  reusable  moulage  (assume  10-15  uses) 

$525-$750 

TOTAL  -  full  manikin 

$452 

HYBRID  HALF-BODY  MANIKIN  +  HUMAN 

Half-body  manikin  (one  trainee/week,  10  year  life) 

$32,500-$40,000 

$82 

Manikin  supplies  &  warranty  as  above 

288 

3  hours  volunteer  time  (E4/E5  median  salary  +  supplements) 

$34/hour 

102 

TOTAL  -  half  manikin 

$472 

Source:  GSA  Schedule 


Opportunities  for  Training  and  Professional  Development  Provided  by  Project 

The  Yale  Team  had  the  opportunity  to  visit  some  DoD  medical  training  sights  as  well  as  to  see 
how  the  simulation  section  in  Tampa,  Florida  was  used  to  train  military  personnel  for  combat. 

Dissemination  of  Information  to  Communities  of  Interest 

This  study  in  its  entirety  will  be  shared  with  the  DoD  in  general,  TATRIC,  the  two  live  tissue 
consortiums,  simulation  researchers,  and  system  dynamics  experts. 

Next  Reporting  Period: 

Nothing  to  Report.  This  is  the  final  report 


4.  IMPACT 

Impact  on  the  the  development  of  principal  disciplines  of  the  project 

This  study  was  conducted  as  a  proof  of  concept  for  Dr.  Terry  Rasch  on  how  system  dynamics 
could  be  employed  to  assess  the  impact  and  nuances  of  funded  DoD  research.  Every  year,  the 
biomedical  division  of  the  DoD  spends  in  excess  of  a  billion  dollars  per  year  on  research.  It  is  not 
known  if  these  funded  studies  are  having  impact  on  soldiers  or  pushing  the  medical  body  of 
knowledge  in  general  forward.  As  system  dynamics  modeling  allows  us  to  look  at  recursive 
relationships,  to  mix  qualitative  and  quantitative  data  together,  to  simulate  out  results  with 
changes  in  practice  or  policy,  it  is  a  superior  method  to  assess  impact  and  return  on  investment 
for  the  DoD’s  portfolios  of  research. 


Impact  on  other  disciplines 

The  results  of  this  study  will  be  presented  to  the  DoD,  TATRIC,  and  Simulation  researchers  in 
general.  This  methodology  can  be  used  in  other  disciplines  as  this  study  suggested.  In  general 
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most  research  and  opinions  of  DoD  physicians  did  not  believe  that  live  tissue  could  be  reduced, 
however,  when  comparing  live  tissue  training  to  simulators,  there  was  not  much  difference.  This 
study  demonstrates  a  mechanism  to  disaggregate  perceptions  from  facts  and  also  how  to  use  the 
most  impactful  combination. 

Impact  on  technology  transfer 

Given  that  this  study  resulted  in  a  simulation  program  that  allows  the  user  to  modify  study 
parameters  and  to  test  out  policies  of  live  tissue  research,  the  system  itself  can  help  those 
experiment  with  various  combinations  to  find  optimal  outcomes.  Furthermore,  this  live  system 
located  on  the  web,  will  be  disseminated  for  use  by  non  DoD  and  DoD  medical  researchers. 

Impact  on  society  beyond  science  and  technology 


Using  live  animals  for  medical  training  has  some  adverse  outcomes  for  the  DoD.  Specifically, 
persons  from  PETA  and  other  animal  rights  activists  are  calling  for  the  DoD  to  end  training  that 
involves  animals.  This  study  allows  us  to  safely  reduce  training  on  animals  and  should  address 
political  issues  with  persons  who  feel  animal  research  is  unethical. 


5.  CHANGES/PROBLEMS 

Changes  in  approach  and  reasons  for  change 

The  largest  challenge  we  experienced  was  we  expected  that  since  we  were  conducting  DoD 
research  that  we  would  have  access  to  military  bases  to  observer  training  of  medical  personnel.  It 
was  not  until  we  involved  a  former  retired  military  person  to  contact  the  correct  people,  we  could 
not  access  the  bases.  After  accessing,  the  study  went  well.  The  initial  goal  of  the  project  was  to 
directly  observe  both  simulation  training  and  live  tissue  training  at  military  sites.  The  primary  site 
for  observation  of  live  tissue  training  was  the  Joint  Special  Operations  Medical  Training  Facility  at 
Fort  Bragg  MSTC  in  North  Carolina. 

Actual  problems  or  delays  and  actions  to  resolve  them 


Since  accessing  the  military  sites  and  DoD  medical  training  took  longer,  our  research  team  had  to 
apply  for  a  No  cost  extensions  (NCE).  The  DoD  willingly  gave  us  the  extension  and  we  were  able 
to  accomplish  our  project  goals  and  objectives. 

Final  NCE  through  9  July  2016  (Appendix)  -  Delivery  Schedule  change  from  10-Sept-2014  to  09- 
Mar-201 6  to  1 0-Sept-201 4  to  09-Jul-201 6 

Changes  that  had  a  significant  impact  on  expenditures 

There  were  no  significant  impact  on  expenditures. 


6.  PRODUCTS 

System  Dynamics  Model:  The  following  link  connects  the  end  user  to  the  live  system  dynamic’s 
model,  https://sims.iseesvstems.eom/psh/emt/#paqe1 
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Return  on  Investment: 


The  aforementioned  live  system  dynamic’s  model  also  includes  the  return  on  investment.  There 
are  several  pages  that  are  accessible  by  the  arrow  in  the  bottom  right  corner.  On  the  final  page, 
the  end-user  must  click  on  the  upper  right  tab  to  reach  the  final  outcomes  and  allows  the  end- 
user  to  experiment  with  the  model  by  changing  parameters  to  see  what  types  of  results  in  terms 
of  training  will  occur  under  varying  conditions  including  return  on  investment. 


7.  PARTICIPANTS  AND  OTHER  COLLABORATING  ORGANIZATIONS 

Individuals  who  have  worked  on  the  project: 

Pis:  Leigh  Evans  and  Lori  Post  were  responsible  for  the  study  outcomes  and  products.  Ms. 
Christal  Esposito  served  as  a  project  manager.  Ms.  Joanne  McGovern  served  as  a  site 
consultant.  Dr.  Peter  Hovmand  conducted  the  Group  Model  Building  exercises.  We  relied  on  two 
groups.  The  first  was  military  physicians  and  the  second  was  simulation  only  physician  trainers,. 


Group  Model  Building  Sessions  (May  2015) 

Military  background: 

a.  Dr.  David  Hile  -  Yale  University 

b.  Dr.  Cynthia  Brandt  -  Yale  University 

c.  Dr.  David  Della-Giustina  -  Yale  University 

d.  Dr.  Paul  Porter  -  Brown  University 

e.  Dr.  Craig  Goolsby  -  USUHS 

f.  Dr.  Joseph  Schmit  -  Baystate  Hospital/Tufts  University 

Simulation  expert  background  -  No  military  training 

a.  Dr.  Leigh  Evans  -  Executive  Director,  Yale  Center  for  Medical  Simulation 

b.  Dr.  Jay  Bonz  -  Director  of  Procedural  Simulation,  Yale  Center  for  Medical  Simulation 

c.  Dr.  Kelly  Dodge  -  Director  of  Resident  Simulation,  Yale  Center  for  Medical  Simulation 

d.  Dr.  Tiffany  Moadel  -  Medical  Simulation  Fellow,  Yale  Center  for  Medical  Simulation 

e.  Dr.  Kimberly  Davis  -  Chief,  Trauma,  Critical  Care  and  Emergency  General  Surgery, 
Department  of  Surgery,  Yale  School  of  Medicine 

f.  Dr.  Scott  Dale  -  Emergency  Medicine  Resident,  Yale-New  Haven  Hospital 

g.  Dr.  Susan  Varga  -  Emergency  Medicine  Resident,  Yale-New  Haven  Hospital 


Change  in  active  support  of  PD/PI(s)  or  senior/key  personnel: 

Other  organizations: 

At  the  writing  of  the  first  proposal  to  the  DoD,  we  were  under  the  assumption  that  we  had  to  use 
an  IDIQ  funding  mechanism.  To  that  end,  we  connected  with  a  health  economist  from  Truven 
Health.  After  beginning  the  project,  we  felt  that  his  skills  were  not  sufficient  to  conduct  a  return  on 
investment.  Since  we  used  a  BAA  mechanism  and  were  free  to  use  another  health  economist,  we 
switched  from  Truven  Health  to  Pacific  Institute  for  Research  and  Evaluation.  Dr.  Ted  Miller  did 
the  final  return  on  investment. 


8.  SPECIAL  REPORTING  REQUIREMENTS 
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Quad  charts 


We  were  required  to  present  initial  findings  to  the  DoD  last  June  201 5,  we  also  filed  monthly 
status  reports  and  finally,  two  quad  charts.  Here  is  our  final  quad  chart. 


Evaluation  of  the  Effectiveness  of  Live  Tissue  Training  versus 
Simulation  Training  and  Stress  Inoculation 

Log  #14154002 
W81XWH-14-C-1392 

PI:  Leigh  Evans,  MD,  Lori  Post,  PhD_ Org:  Yale  University/ Emergency  Medicine  Award  Amount:  417,751 


Study/Product  Aims 

The  assessment  benefits  the  Department  of  Defense  by: 

1.  Establishing  a  system  dynamics  proofof  concept  before  engaging  in  a  full  study  to  determine 
if  simulation  can  replace  live  animal  training  in  some  procedures. 


Documant 

review 


Preparing 

preliminary 

model 


Group 

Model 

Building 

Sessions 


Synthesis 


Review  by 
participants 
(member  checking) 


Proof -of- 
Concept 
Simulation 
Model 


2.Establish  a  Return  on  Investment  proof  of  concept  before  conducting  a  full  cost  study  to 
determine  if  specific  modes  oftraining  have  better  return  on  investment. 

Approach 

1.  Develop  a  conceptual  systems  dynamic  model  of  live  tissue  vs.  simulation  training. 

•  Conduct  literature  review 

•  Develop  seed  model  fromthe  literature  and  transcripts  of  the  Live  Tissue  Vs. 
Simulation  Training  Consortium  Meeting  in  Tampa  FL 

•  Recruit  and  convene  civilian  and  military  physicians  to  review  the  system 
dynamics 

•  Develop  the  systems  dynamic  model 

2. Conducting  a  formative  evaluation  -  Define  the  scope  of  the  problem,  identify  possible 
solutions,  determine  if  there  are  any  barriers  which  would  prohibit  the  implementation  of  the 
solution. 

3. Determining  the  return  on  investment  (ROI>-  Describe  each  training  alternative,  determine 
costs  affiliated  with  each  approach,  calculate  costsand  ROI. 


Next  steps 

Completed  the  quantitative  analysis  of  the  DoD  LTR/Simulation  Consortium 
and  literature  review.  Developed  the  seed  model  and  convened  civilian  and 
military  physicians  to  refine  the  system  dynamic  model.  Conducted  return 
on  investment  and  system  dynamics  model 


Timeline  and  Cost 


Goals/Milestones 


Activities 


Develop  and 
testa 

conceptual 

systems 

dynamics 

model 

Conduct 

formative 

evaluation 


Determining 
the  return  on 
investment 
(ROI)- 


Esti  mated  $36,591  $166,021  $66,962  $34,859  $56,659  $56,659 


CY  201 5  Goal  1  -  Initiate  the  development  of  a  conceptual  systems  dynamic  model  of  live 
tissuevs.  simulation  training. 

□  Develop  seed  model  fromthe  literature  and  transcripts  of  the  Live  Tissue  Vs.  Simulation 
Training  Consortium  Meeting  in  Tampa  FL 

□  Recruit  and  convene  civilian  and  military  physicians  to  review  the  system's  dynamics 

□  Validate  seed  model  adjust  as  required 

CY  2015  -16  Goal  2  -  Conduct  formative  evaluation 

□  Conduct  site  visit  to  military  MTF  to  evaluate  the  model  and  build  upon  it 

□  Define  the  scope  of  the  issues  surrounding  live  tissue  vs.  simulation  training  and  compare  the 
advantages/disad  vantages  of  both 

□  Identify  possible  solution  sets,  determine  if  there  are  any  barriers  which  would  prohibit  implementation 

□  Finalize  and  run  the  system  dynamic  model  to  determine  how  many  lives  will  be  saved  over  time 
utilizing  Live  Tissue  Vs.  Simulation  T raining 

CY  2016  -17  Goal  -  Determining  the  return  on  investment  (ROI) 

□  Identify  all  cost  associated  with  Live  Tissue  Vs.  Simulation  Training 

□  Execute  a  live  system  dynamics  model  combining  data  and  group  model  building  and  observation  data 
Comments/Chal  lenges/l  ssues/Concems 

A  no  cost  extension  for  15  months  was  given  due  to  delays  m  the  start  of  the  project  due  to  sequestration  and 
access  to  military  base  training. 

Budget  Expenditure  to  Date 
Projected  Expenditure:  5417,751 
Actual  Expenditure:  5417,751 


Budeet 


9.  APPENDICES 

Given  that  the  main  product  of  this  study  is  an  on  line  system  dynamic’s  model  of  live  tissue 
training  vs.  simulator  training,  we  do  not  have  an  appendix.  Rather,  we  have  a  link  to  the  website. 
Furthermore,  we  are  in  the  process  of  publishing  our  findings. 

https://sims.iseesvstems.eom/psh/emt/#paqe1 
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